Hybrid cooling device for acceleration hardware

ABSTRACT

Described herein is a hybrid cooling device and a cooling method that use a combination of phase change cooling and air cooling. The hybrid cooling device includes a closed loop two phase system, one or more fans, and an assembly clamp. The two phase system further includes a cold plate, an integrated channel, and a radiator, and a pressure sensor. The cold plate can include phase change fluid for extracting heat from electronics on a printed circuit board sandwiched between the cold plate and the assembly clamp. The one or more fans can be used to create airflows for cooling both the cold plate and the radiator. The pressure sensor can be used to control the operation of the hybrid cooling device, which can be deployed in different system environments and server configurations.

FIELD OF THE INVENTION

Embodiments of the present disclosure relate generally to coolingsystems. More particularly, embodiments of the disclosure relate to ahybrid cooling device and a hybrid cooling method that use both phasechange cooling and air cooling.

BACKGROUND

A high power density device is a computing device that is packaged withhigh performance processors (e.g., such as GPU, ASIC, heterogeneouscomputing based IC chip or chiplet). Such high power density devices areincreasingly popular due to the continuous high computing need. A highpower density device tends to generate a large amount of heat and isoften integrated into a server chassis. Therefore, for a high powerdensity device to function properly, a proper thermal environment forservers, racks, and data center facility is needed.

Although liquid cooling can be a promising cooling solution for highpower density devices, particular when the power budget for a singlechip exceeds a threshold (e.g., 400 W), the required accompanyingfacility can be a bottleneck, because such a liquid cooling solution hascertain requirements for supply inlet temperatures, flow rates andpressures that exceed the capability of a typical data center. Even if adata center facility can be developed to meet the requirements, the costwould be too high.

Further complicating the problem is that many high performance hardwarecomponents are connected through a peripheral component interconnectexpress (PCIe) expansion bus. A liquid cooling solution for suchhardware components and packages requires completely differentarchitecture compared to Mezzanine connector based cards.

Previous cooling solutions for the PCIE based electronics focus ondesktop products, rather than on hyper scale cloud data centers. Suchcooling solutions may not be feasible for integration into servers in acloud data center. Further, these solutions may be unscalable,inversatile, not reliable enough, or too costly. In addition, most ofthe solutions are air cooling based, which may not satisfy theconstantly increasing power density.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are illustrated by way of example and notlimitation in the figures of the accompanying drawings in which likereferences indicate similar elements.

FIGS. 1A-1B show hardware for a hybrid cooling device according to oneembodiment.

FIGS. 2A-2B show a hardware design with fans in the hybrid coolingdevice according to one embodiment.

FIGS. 3A-3C show the hybrid cooling device combined with electronics ona printed circuit board according to one embodiment.

FIGS. 4A-4C show thermal management within the hybrid cooling deviceaccording to one embodiment.

FIGS. 5A-5B show an overall system level use of the hybrid coolingdevice in a server according to one embodiment.

FIG. 6 shows the hybrid cooling device as described in FIG. 2 beingdeployed in a server chassis according to one embodiment.

FIGS. 7A-7B show an operation control of the hybrid cooling deviceaccording to one embodiment.

FIG. 8 is a flow diagram illustrating a control flow process 800 for thehybrid cooling device according to one embodiment.

FIG. 9 illustrates a method 900 of cooling a heterogeneous computingarchitecture according to one embodiment.

FIG. 10 is block diagram illustrating an electronic rack according toone embodiment.

DETAILED DESCRIPTION

Various embodiments and aspects of the inventions will be described withreference to details discussed below, and the accompanying drawings willillustrate the various embodiments. The following description anddrawings are illustrative of the invention and are not to be construedas limiting the invention. Numerous specific details are described toprovide a thorough understanding of various embodiments of the presentinvention. However, in certain instances, well-known or conventionaldetails are not described in order to provide a concise discussion ofembodiments of the present inventions.

Reference in the specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin conjunction with the embodiment can be included in at least oneembodiment of the invention. The appearances of the phrase “in oneembodiment” in various places in the specification do not necessarilyall refer to the same embodiment.

According to various embodiments, described herein is a hybrid coolingdevice and a cooling method that use a combination of phase changecooling and air cooling. The hybrid cooling device includes a closedloop two phase system, one or more fans, and an assembly clamp. The twophase system further includes a cold plate, an integrated channel, and aradiator as a condenser. The cold plate can include phase change fluidfor extracting heat from electronics on a printed circuit board (PCB)sandwiched between the cold plate and the assembly clamp. The one ormore fans can be used to create airflows for cooling both theelectronics on the PCB and the radiator. A pressure sensor and atemperature sensor can be used to control the operation of the hybridcooling device, which can be integrated into different systemenvironments and server configurations.

In one embodiment, the hybrid cooling device further includes a deviceframe, to which the radiator, the integrated channel, and the cold plateare attached. Further, the hybrid cooling device can include an adaptingstiffener positioned between the cold plate and the electronics on thePCB, and one or more elastic channels. The adapting stiffener and theone or more elastic channels operate in conjunction to maintain properpressure on the electronics on the PCB.

In one embodiment, the hybrid cooling device further includes a movingaxis in the one of the elastic channels, and one end of the assemblyclamp is inserted into the elastic channels through the moving axis suchthat the end of the assembly clamp is moveable on the elastic channel.This elastic channel can provide forces on the moving axis on both sideshorizontally to properly fix the PCB at a particular position within thehybrid cooling device.

In one embodiment, the electronics on the PCB can include one or more ofa chip or a power electronics, and wherein the PCB where the electronicsinstalled on are connected by a peripheral component interconnectexpress (PCIe) bus to a server main PCB.

In one embodiment, the integrated channel includes a vapor line and aliquid line, the liquid line for passing liquid from the radiator to thecold plate, and the vapor line for passing vapor from the cold plate tothe radiator. In one embodiment, the vapor line and liquid line may bedesigned in different physical dimensions for better performance.

In one embodiment, each of the one or more fans can be a fan integratedinto the hybrid cooling device or a separate fan. The airflows createdby the one or more fans pass through the PCB through a first dedicatedchannel, and pass through the radiator through a second dedicatedchannel.

In one embodiment, the hybrid cooling device can include a temperaturesensor and a pressure sensor to control the operation of the hybridcooling device. In one embodiment, the hybrid cooling device can includeonly a pressure sensor, and the pressure sensor is pre-integrated on thevapor line in the hybrid cooling device.

In one embodiment, the hybrid cooling method can be deployed todifferent chassis, e.g., blade servers. Further, multiple electronics ona PCB or multiple PCBs can be packaged within the hybrid cooling device.A variety of clamping methods can be used for sandwiching the PCBs.

The hybrid cooling device can be deployed in any server or chassisenvironment, and is compatible with different heterogeneous hardwareconfigurations for complex and multiple heterogeneous computingworkloads. As such, the hybrid cooling device is scalable andinteroperable for different server system designs and configurations,including different heterogeneous hardware expansions. In addition, thesolution is highly efficient since fluid is self-driven with phasechange technologies.

The various embodiments provide a solution for hyperscale data centersapplications and corresponding servers in a cloud environment, as wellas for edge computing system, either in edge cluster or edge devices.The cooling solution described in the various embodiments can be usedfor cooling high power density electronics. With a complete packingmethod for designing hybrid cooling devices, the cooling solution can beconfigured for different hybrid designs such as phase change with air inparallel, phase change liquid cooling only and so on.

FIGS. 1A-1B show a hybrid cooling device according to one embodiment.FIG. 1A shows a front view of the hybrid cooling device, and FIG. 1Bshows a side view of the hybrid cooling device.

As shown, the hybrid cooling device include a radiator 101, anintegrated channel 103, a cold plate 105, an assembly clamp 107, and adevice frame 109. The radiator 101, the integrated channel 103, and thecold plate 105 can be combined into a single unit. The single unitconstitutes the main component of the hybrid cooling device.

However, despite being a single unit, the integral designs for the threecomponents 101, 103 and 105 can be different depending on actualimplementations and specific requirements of different users.

The device frame 109 can be a hardware frame, to which the radiator 101,the integrated channel 103, and the cold plate 105 are attached. Theintegrated channel 103 can include a liquid line and a vapor line forconnecting the radiator and the cold plate. The assembly clamp 107,which is described in detail below, can be used to hold electronics on aprinted circuit board (PCB) with proper pressure.

FIGS. 2A-2B further show the hybrid cooling device according to oneembodiment. FIG. 2A shows a front view of the hybrid cooling device, andFIG. 1B shows a side view of the hybrid cooling device.

As shown, the hybrid cooling device can include a fan 201. The fan 201and the single unit described above, provides a hybrid coolingenvironment for a printed circuit board (PCB) 203 with high powerdensity electronics installed thereon.

In one embodiment, the PCB 203 be an acceleration PCB that includesmultiple hardware components to speed up data communication, storage andretrieval, encryption and decryption, mathematical operations, graphics,and web page viewing, etc. The PCB 203 can be attached to the cold plate105. Both the radiator 101 and the PCB 203 can be air cooled by the fan201. The solution shown in FIG. 2A can be understood as that the fansare integrated together as one unit. This means the fan design isoptimized in terms of locations, fan selection and airflow management.

In one embodiment, the structural layout of the hybrid cooling deviceenables the fan 201 to blow direct or indirect airflows towards both theradiator 101 and the electronics on the PCB 203. As such, the fan 201can provide direct air cooling and indirect air cooling. The fan 201 canbe an integrated unit of the hybrid cooling device, or a separate moduleattached to the hybrid cooling device.

FIGS. 3A-3C show the hybrid cooling device combined with an PCBaccording to one embodiment. FIG. 3A illustrates an overall structure ofthe hybrid cooling device, and FIGS. 3B and 3C provides additionalimplementation details.

In FIG. 3A, the PCB 203 can have different types of chips or powerelectronics installed thereon. An adapting stiffener 305 can be usedbetween the cold plate 105 and the chips or power electronics on the PCB203 to ensure that the hybrid cooling device is properly assembled. Inone embodiment, the adapting stiffener 305 can be resilient, and can bemade of elastic material to accommodate the different heights ofelectronics installed on the PCB 203.

The hybrid cooling device further includes a connection bus 301 used toconnect the different electronics on the PCT 203. The connection bus 301can be a peripheral component interconnect express (PCIe) bus, which isan interface standard for connecting high-speed components.

In FIG. 3A, the cold portion of the hybrid cooling device can cover allelectronics on the PCB 203 such that they all can be cooled by the coldplate 105.

FIG. 3B illustrates the assembly clamp 107 in detail. The assembly clamp107 includes four parts: an elastic channel 307, a moving axis 309, andtwo assembly shafts 311 and 313. In this figure, the clamp assembly 107is not locked. As shown, one end of the assembly shaft 311 is insertedinto the elastic channel 307 through the moving axis 309, and therefore,this end is moveable on the elastic channel 307. The elastic channel 307can provide forces on the moving axis 309 on both sides along thehorizontal direction to ensure proper fixing of the hybrid coolingdevice in terms of the PCB 203, as well as ensuring proper thermalcontacting between the electronics and the stiffener.

FIG. 3C shows a view of the hybrid cooling device when it is locked. Asshown, in addition to the elastic channel 307, the hybrid cooling devicecan include another elastic channel 315 on the assembly shaft 315. Thetwo assembly shafts 311 and 313 are connected together to form theassembly clamp 107.

The assembly clamp 107 can be locked and unlocked by turning around themoving axis 309. When the assembly clamp 107 is locked, the PCB 203, thechips 303 (also referred to as electronics) on the PCB 203, and theadapting stiffener 305 can be sandwiched between the cold plate 105 andthe assembly shaft 107. Further, when the assembly clamp 107 is locked,the two elastic channels 307 and 315 can ensure that proper pressure beexerted on the chips 303 and the PCB 203 to avoid damages, and toprevent them from malfunctions. The elastic channels 307 and 315 canalso ensure proper thermal contacting between the cold plate 105 and thechips 303.

FIGS. 4A-4C show thermal management within the hybrid cooling device anda hybrid environment according to one embodiment. FIG. 4A shows a frontview of the hybrid cooling device, and FIGS. 4B-4C show a front view ofthe hybrid cooling device.

As shown in FIG. 4A, the cold plate 105 and the radiator 101 areconnected by a liquid line 403 and a vapor line 404, each of which is apipe that vapor, air, or fluid can pass through. The liquid line 403 andthe vapor line 404 form the integrated channel 103 described in FIG. 2A.

In FIG. 4A, a phase change 405 can occur within the cold plate 105 as aresult of heat being extracted from the electronics/chips on the PCB203. Fluid from the radiator 101 can pass through the liquid line 403 tothe cold plate 105, where the fluid changes its phase to vapor 404 afterabsorbing the heat extracted from the chips on the PCB 203. The vaporcarrying latent heat does not vary its temperature due to the phasechange. The phase change causes a pressure increase in the cold plate105, and the increased pressure elevates the vapor to the radiator 101through the vapor line 404.

The radiator 101 can function as a condensing unit to condense the vaporelevated from the cold plate 105 back to liquid by extracting its latentheat from the vapor. The liquid can return to the cold plate driven bythe gravity force.

FIG. 4B shows airflows 407 and 409 that are created by a fan, e.g., thefan 201 illustrated in FIG. 2A and FIG. 2B. The fan can create theairflows 407 and 409 either by pumping or pulling air. The fan can be oneither side of the hybrid cooling device.

In one embodiment, the airflows 407 can pass through the radiator 101 toassist the radiator 101 in condensing vapor to liquid, and the airflows409 can pass through the chips or electronics on the PCB 203 to provideair cooling to the chips or electronics on the PCB 203. In FIG. 4B, thehybrid cooling device uses dedicated channels to manage and optimize theairflows 407 and 409.

Alternatively, FIG. 4C shows another design for thermal management,where airflows 411 pass through the radiator 101 and the PCB 203 and theelectronics on the PCB 203 in parallel since no dedicated channels forairflows 411 are used.

FIG. 4B and FIG. 4C show different airflow management within the hybridcooling device with different fan implementations. This can beunderstood as how the full set of the hybrid cooling device is used andconfigured to create different hybrid cooling environments. FIG. 4Bshows that the portion of inlet airflow is used for cooling the radiatorto condense the vapor back to liquid and the other portion is used forcooling the other air cooled electronics on the PCB directly. The heatedair is converged to the dedicated channel driven by the fan. While inFIG. 4C, the two portions of airflows form separate paths.

FIGS. 5A-5B show an overall system level use of the hybrid coolingdevice according to one embodiment. The figures show that a hybridcooling device 501 can be integrated into a server chassis 507, wherethe hybrid cooling device 501 can adapt to the environment of the serverchassis 507 and take advantage of the existing server chassisenvironment and structure.

In FIG. 5A, the hybrid cooling device 501 includes a phase changecooling portion that occurs in a cold plate 507 and a dedicated fan 505for cooling an acceleration PCB 503 and electronics installed thereon.The dedicated fan 505 can be a cross flow fan, and can be used to assistin generating the airflows shown in FIG. 4 .

As further shown, the server chassis 507 can include a server PCB 505and a chassis fan 509 mounted on the right side of the hybrid coolingdevice 501. The chassis fan 509, as part of the existing server chassisstructure, can function as the primary air mover. Thus, the hybridcooling device 501 can take advantage of the existing server chassisstructure.

In FIG. 5B, an additional fan 511 is integrated into the hybrid coolingdevice 501 for enhancing airflows. The additional fan 511 can be usedfor redundancy since the server chassis 507 may not be dedicated for theacceleration PCB 503. The additional fan 511 can further enhance systemperformance.

FIG. 6 shows the hybrid cooling device as described in FIG. 2 beingdeployed in a server chassis according to one embodiment.

In this embodiment, unlike the embodiments illustrated in FIGS. 5A-5B,the hybrid cooling device 601 is fully disaggregated from the serverchassis 507 in terms of airflow management, which means that no serverfan is needed.

In the various embodiments described above, the hybrid cooling device inFIGS. 5A-5B and 6 can be reconfigured by adding additional features totake advantage of the environment in the server chassis 507.

FIGS. 7A-7B show how the hybrid cooling device is controlled accordingto one embodiment.

As shown, the hybrid cooling device can include two sensors. A pressuresensor 701 can be attached to the vapor line 403 to measure the pressureof the vapor passing through the vapor line 404. A temperature sensor703 can be provided in the cold plate to measure the temperature of thecold plate. These two sensors 701 and 703 are decoupled from any of theelectronics on the PCB 503. The decoupling can significantly increasethe adaptability and reliability of the cooling solution. In oneembodiment, the temperature sensor can be a sensor in the chip package,such as a sensor for measuring the case temperatures. In this case, onlythe pressure is needed on the hybrid cooling device for the purpose ofcontrolling the operation of the hybrid cooling device.

In one embodiment, the two sensors 701 and 703 are used for controllingthe fan or fans of the hybrid cooling device only, and the devicecontrol applies to only the hardware of the device, and does not applyto the PCBs 503 and 505 and the electronics on the two PCBs. Such adesign can increase the hybrid cooling device's deployability,tunability, and interoperability. The design aims to simplify the systemintegration and tuning procedures, which means plug and play.

FIG. 8 is a flow diagram illustrating a control flow process 800 for thehybrid cooling device according to one embodiment.

As shown in FIG. 8 , a temperature sensor and a pressure sensor are usedfor controlling the operation of the hybrid cooling device, whichincludes a main fan and a secondary fan. The flow control process 400may be performed by processing logic which may include software,hardware, or a combination thereof.

In operation 801, the processing logic initiates the temperature sensorto measure the temperature inside the cold plate in the hybrid coolingdevice, and initiates the pressure sensor to measure the pressure of thevapor passing through the vapor line.

In operation 803, the processing logic determines whether the measuredtemperature is under a predetermined threshold (i.e., T_(case-design)).

In operation 805, if the measured temperature is not under thepredetermined threshold, the processing logic can send commands to runthe main fan in the hybrid cooling device to its maximum speed.

In operation 806, the processing logic determines whether the measuredtemperature has decreased under the predetermined threshold due to theblowing of the main fan at its maximum speed.

In operation 807, the measured temperature has decreased under thethreshold hold. The processing logic continues monitoring thetemperature, and also uses the measured pressure to control theoperation of the hybrid cooling device.

In operation 808, the measured temperature has not decreased under thethreshold hold, and the processing logic runs the secondary fan to itsmaximum speed.

In operation 809, the processing logic determines whether the measurespressure has increased.

In operation 811, the processing logic determines that the measuredpressure has not increased and accordingly decreases the speed of themain fan.

In operation 813, the processing logic determines that the measuredpressure has increased, and accordingly increases the speed of the mainfan if the main fan is not running at its maximum speed.

In operation 815, the processing logic determines whether the measuredtemperature exceeds the predetermined threshold. If so, the processinglogic will monitor the measured temperature to determine if it decreasesunder the predetermined threshold; otherwise, the processing logic willcheck if the measured pressure has increased.

FIG. 9 illustrates a method 900 of cooling a heterogeneous computingarchitecture according to one embodiment.

As shown in FIG. 9 , in block 901 a phase change system that includes acold plate, a radiator, and an integrated channel connecting the coldplate and the radiator. In block 903 an assembly clamp is provided toposition electronic hardware to be cooled between the assembly clamp andthe cold plate. In block 903, one or more fans are provided. The fanscan be integrated with the cold plate and the radiator or can beseparate fans. In block 907, the phase change system is used to cool theelectronics hardware, and the airflows created by the one or more fansare used to cool the radiator and the electronic hardware.

FIG. 10 is block diagram illustrating an electronic rack according toone embodiment. Electronic rack 1000 may represent any of the electronicracks of a data center. Referring to FIG. 10 , according to oneembodiment, electronic rack 1000 includes, but is not limited to, CDU1001, rack management unit (RMU) 1002, and one or more server chassis1003A-1003E (collectively referred to as server chassis 1003). Serverchassis 1003 can be inserted into an array of server slots (e.g.,standard shelves) respectively from frontend 1004 or backend 1005 ofelectronic rack 1000. Note that although there are five server chassis1003A-1003E shown here, more or fewer server chassis may be maintainedwithin electronic rack 1000. Also note that the particular positions ofCDU 1001, RMU 1002, and/or server chassis 1003 are shown for the purposeof illustration only; other arrangements or configurations of CDU 1001,RMU 1002, and/or server chassis 1003 may also be implemented. In oneembodiment, electronic rack 1000 can be either open to the environmentor partially contained by a rack container, as long as the cooling fanscan generate airflows from the frontend to the backend.

In addition, for at least some of the server chassis 1003, an optionalfan module (not shown) is associated with the server chassis. Each ofthe fan modules includes one or more cooling fans. The fan modules maybe mounted on the backends of server chassis 1003 or on the electronicrack to generate airflows flowing from frontend 1004, traveling throughthe air space of the sever chassis 1003, and existing at backend 1005 ofelectronic rack 1000.

In one embodiment, CDU 1001 mainly includes heat exchanger 1011, liquidpump 1012, and a pump controller (not shown), and some other componentssuch as a liquid reservoir, a power supply, monitoring sensors and soon. Heat exchanger 1011 may be a liquid-to-liquid heat exchanger. Heatexchanger 1011 includes a first loop with inlet and outlet ports havinga first pair of liquid connectors coupled to external liquidsupply/return lines 131-132 to form a primary loop. The connectorscoupled to the external liquid supply/return lines 131-132 may bedisposed or mounted on backend 1005 of electronic rack 1000. The liquidsupply/return lines 131-132, also referred to as room liquidsupply/return lines, may be coupled to an external cooling system (e.g.,a data center room cooling system).

In addition, heat exchanger 1011 further includes a second loop with twoports having a second pair of liquid connectors coupled to liquidmanifold 1025 (also referred to as a rack manifold) to form a secondaryloop, which may include a supply manifold (also referred to as a rackliquid supply line or rack supply manifold) to supply cooling liquid toserver chassis 1003 and a return manifold (also referred to as a rackliquid return line or rack return manifold) to return warmer liquid backto CDU 1001. Note that CDUs 1001 can be any kind of CDUs commerciallyavailable or customized ones. Thus, the details of CDUs 1001 will not bedescribed herein.

Each of server chassis 1003 may include one or more IT components (e.g.,central processing units or CPUs, general/graphic processing units(GPUs), memory, and/or storage devices). Each IT component may performdata processing tasks, where the IT component may include softwareinstalled in a storage device, loaded into the memory, and executed byone or more processors to perform the data processing tasks. Serverchassis 1003 may include a host server (referred to as a host node)coupled to one or more compute servers (also referred to as computingnodes, such as CPU server and GPU server). The host server (having oneor more CPUs) typically interfaces with clients over a network (e.g.,Internet) to receive a request for a particular service such as storageservices (e.g., cloud-based storage services such as backup and/orrestoration), executing an application to perform certain operations(e.g., image processing, deep data learning algorithms or modeling,etc., as a part of a software-as-a-service or SaaS platform). Inresponse to the request, the host server distributes the tasks to one ormore of the computing nodes or compute servers (having one or more GPUs)managed by the host server. The compute servers perform the actualtasks, which may generate heat during the operations.

Electronic rack 1000 further includes optional RMU 1002 configured toprovide and manage power supplied to servers 1003, and CDU 1001. RMU1002 may be coupled to a power supply unit (not shown) to manage thepower consumption of the power supply unit. The power supply unit mayinclude the necessary circuitry (e.g., an alternating current (AC) todirect current (DC) or DC to DC power converter, battery, transformer,or regulator, etc.,) to provide power to the rest of the components ofelectronic rack 1000.

In one embodiment, RMU 1002 includes optimization module 1021 and rackmanagement controller (RMC) 1022. RMC 1022 may include a monitor tomonitor operating status of various components within electronic rack1000, such as, for example, computing nodes 1003, CDU 1001, and the fanmodules. Specifically, the monitor receives operating data from varioussensors representing the operating environments of electronic rack 1000.For example, the monitor may receive operating data representingtemperatures of the processors, cooling liquid, and airflows, which maybe captured and collected via various temperature sensors. The monitormay also receive data representing the fan power and pump powergenerated by the fan modules and liquid pump 1012, which may beproportional to their respective speeds. These operating data arereferred to as real-time operating data. Note that the monitor may beimplemented as a separate module within RMU 1002.

Based on the operating data, optimization module 1021 performs anoptimization using a predetermined optimization function or optimizationmodel to derive a set of optimal fan speeds for the fan modules and anoptimal pump speed for liquid pump 1012, such that the total powerconsumption of liquid pump 1012 and the fan modules reaches minimum,while the operating data associated with liquid pump 1012 and coolingfans of the fan modules are within their respective designedspecifications. Once the optimal pump speed and optimal fan speeds havebeen determined, RMC 1022 configures liquid pump 1012 and cooling fansof the fan modules based on the optimal pump speeds and fan speeds.

As an example, based on the optimal pump speed, RMC 1022 communicateswith a pump controller of CDU 1001 to control the speed of liquid pump1012, which in turn controls a liquid flow rate of cooling liquidsupplied to the liquid manifold 1025 to be distributed to at least someof server chassis 1003. Similarly, based on the optimal fan speeds, RMC1022 communicates with each of the fan modules to control the speed ofeach cooling fan of the fan modules, which in turn control the airflowrates of the fan modules. Note that each of fan modules may beindividually controlled with its specific optimal fan speed, anddifferent fan modules and/or different cooling fans within the same fanmodule may have different optimal fan speeds.

Note that the rack configuration as shown in FIG. 10 is shown anddescribed for the purpose of illustration only; other configurations orarrangements may also be applicable. For example, CDU 1001 may be anoptional unit. The cold plates of server chassis 1003 may be coupled toa rack manifold, which may be directly coupled to room manifolds 131-132without using a CDU. Although not shown, a power supply unit may bedisposed within electronic rack 1000. The power supply unit may beimplemented as a standard chassis identical or similar to a severchassis, where the power supply chassis can be inserted into any of thestandard shelves, replacing any of server chassis 1003. In addition, thepower supply chassis may further include a battery backup unit (BBU) toprovide battery power to server chassis 1003 when the main power isunavailable. The BBU may include one or more battery packages and eachbattery package include one or more battery cells, as well as thenecessary charging and discharging circuits for charging and dischargingthe battery cells.

In one embodiment, the cooling devices disposed in each of the serverchassis as shown may represent any cooling device described throughoutthis application.

In the foregoing specification, embodiments of the disclosure have beendescribed with reference to specific exemplary embodiments thereof. Itwill be evident that various modifications may be made thereto withoutdeparting from the broader spirit and scope of the disclosure as setforth in the following claims. The specification and drawings are,accordingly, to be regarded in an illustrative sense rather than arestrictive sense.

As previously explained, an embodiment of the disclosure may be (orinclude) a non-transitory machine-readable medium (such asmicroelectronic memory) having stored thereon instructions, whichprogram one or more data processing components (generically referred tohere as a “processor”) to perform airflow management operations, such ascontrolling fan speed of one or more fans of the battery module (and/orBBU shelf). In other embodiments, some of these operations might beperformed by specific hardware components that contain hardwired logic.Those operations might alternatively be performed by any combination ofprogrammed data processing components and fixed hardwired circuitcomponents of any of the battery modules described herein.

While certain aspects have been described and shown in the accompanyingdrawings, it is to be understood that such aspects are merelyillustrative of and not restrictive on the broad disclosure, and thatthe disclosure is not limited to the specific constructions andarrangements shown and described, since various other modifications mayoccur to those of ordinary skill in the art. The description is thus tobe regarded as illustrative instead of limiting.

In some aspects, this disclosure may include the language, for example,“at least one of [element A] and [element B].” This language may referto one or more of the elements. For example, “at least one of A and B”may refer to “A,” “B,” or “A and B.” Specifically, “at least one of Aand B” may refer to “at least one of A and at least one of B,” or “atleast of either A or B.” In some aspects, this disclosure may includethe language, for example, “[element A], [element B], and/or [elementC].” This language may refer to either of the elements or anycombination thereof. For instance, “A, B, and/or C” may refer to “A,”“B,” “C,” “A and B,” “A and C,” “B and C,” or “A, B, and C.”

What is claimed is:
 1. A hybrid cooling device, comprising: a phasechange system that includes a cold plate, a radiator, and an integratedchannel connecting the cold plate and the radiator; an assembly clamp toposition electronic hardware to be cooled between the assembly clamp andthe cold plate, wherein the radiator is positioned above the cold plate,and wherein the cold plate is positioned vertically to be attached tothe electronic hardware when the assembly clamp clamps onto the coldplate; and one or more fans to provide air cooling the radiator and theelectronic hardware, wherein the electronic hardware includes a printedcircuit board (PCB) and an electronic device packaged thereon, theelectronic device including one or more of a chip or a power electronic,and wherein the phase change system, the assembly clamp and the one ormore fans together with the electronic device packaged on the PCB can beinserted into a peripheral bus as an integrated peripheral device. 2.The hybrid cooling device of claim 1, further comprising: a deviceframe, wherein the radiator, the integrated channel, and the cold plateare attached to the device frame.
 3. The hybrid cooling device of claim1 further comprising: an adapting stiffener positioned between the coldplate and the electronic hardware; an elastic channel; wherein theadapting stiffener and the elastic channel operate in conjunction tomaintain proper pressure on the electronic hardware.
 4. The hybridcooling device of claim 3, further comprising: a moving axis in theelastic channel; wherein one end of the assembly clamp is inserted intothe elastic channel through the moving axis such that the end of theassembly clamp is moveable on the elastic channel.
 5. The hybrid coolingdevice of claim 4, wherein the elastic channel provides forces on themoving axis on both sides horizontally to properly fix the electronichardware within the hybrid cooling device.
 6. The hybrid cooling deviceof claim 1, wherein the integrated channel includes a vapor line and aliquid line, the liquid line for passing liquid from the radiator to thecold plate, and the vapor line for passing vapor from the cold plate tothe radiator.
 7. The hybrid cooling device of claim 1, wherein each ofthe one or more fans is integrated into the hybrid cooling device or aseparate fan.
 8. The hybrid cooling device of claim 1, wherein airflowscreated by the one or more fans pass through the cold plate through afirst dedicated channel, and pass through the radiator through a seconddedicated channel.
 9. The hybrid cooling device of claim 4, furthercomprising: a temperature sensor; a pressure sensor; wherein thetemperature sensor and the pressure sensor are used to control anoperation of the hybrid cooling device.
 10. A server chassis,comprising: a hybrid cooling device including: a phase change systemthat includes a cold plate, a radiator, and an integrated channelconnecting the cold plate and the radiator, an assembly clamp toposition electronic hardware to be cooled between the assembly clamp andthe cold plate, wherein the radiator is positioned above the cold plate,and wherein the cold plate is positioned vertically to be attached tothe electronic hardware when the assembly clamp clamps onto the coldplate, and one or more fans to provide air cooling the radiator and theelectronic hardware, wherein the electronic hardware includes a printedcircuit board (PCB) and an electronic device packaged thereon, andwherein the phase change system, the assembly clamp and the one or morefans together with the electronic device packaged on the PCB can beinserted in to a peripheral bus as an integrated peripheral device; anda chassis fan to provide an airflow to cool the server chassis and thehybrid cooling device.
 11. The server chassis of claim 10, wherein theelectronic device includes one or more of a chip or a power electronics,and wherein the peripheral bus is a peripheral component interconnectexpress (PCIe) bus assembled with the hybrid cooling device.
 12. Theserver chassis of claim 10, wherein the hybrid cooling device furthercomprise: a device frame, wherein the radiator, the integrated channel,and the cold plate are attached to the device frame.
 13. The serverchassis of claim 12, wherein the hybrid cooling device further comprise:an adapting stiffener positioned between the cold plate and theelectronic hardware; an elastic channel; wherein the adapting stiffenerand the elastic channel operate in conjunction to maintain properpressure on the electronic hardware.
 14. The server chassis of claim 13,wherein the hybrid cooling device further comprise: a moving axis in theelastic channel; wherein one end of the assembly clamp is inserted intothe elastic channel through the moving axis such that the end of theassembly clamp is moveable on the elastic channel.
 15. The serverchassis of claim 14, wherein the elastic channel provides forces on themoving axis on both sides horizontally to properly fix the electronichardware within the hybrid cooling device.
 16. The server chassis ofclaim 10, wherein the integrated channel includes a vapor line and aliquid line, the liquid line for passing liquid from the radiator to thecold plate, and the vapor line for passing vapor from the cold plate tothe radiator.
 17. The server chassis of claim 10, wherein each of theone or more fans is integrated into the hybrid cooling device or aseparate fan.
 18. The server chassis of claim 10, wherein airflowscreated by the one or more fans pass through the cold plate through afirst dedicated channel, and pass through the radiator through a seconddedicated channel.
 19. The server chassis of claim 14, wherein thehybrid cooling device further comprise: a temperature sensor; a pressuresensor; wherein the temperature sensor and the pressure sensor are usedto control an operation of the hybrid cooling device.
 20. An electronicrack, comprising: a plurality of server chassis, each server chassisincluding: a hybrid cooling device comprising: a phase change systemthat includes a cold plate, a radiator, and an integrated channelconnecting the cold plate and the radiator, an assembly clamp toposition electronic hardware to be cooled between the assembly clamp andthe cold plate, wherein the radiator is positioned above the cold plate,and wherein the cold plate is positioned vertically to be attached tothe electronic hardware when the assembly clamp clamps onto the coldplate, and one or more fans to provide air cooling the radiator and theelectronic hardware, wherein the electronic hardware includes a printedcircuit board (PCB) and an electronic device packaged thereon, theelectronic device including one or more of a chip or a power electronic,and wherein the phase change system, the assembly clamp and the one ormore fans together with the electronic device packaged on the PCB can beinserted into a peripheral bus as an integrated peripheral device; and achassis fan to provide an airflow to cool the server chassis and thehybrid cooling device.